Persian Viseme Classification Using Interlaced Derivative Patterns and Support Vector Machine

Viseme (Visual Phoneme) classification and analysis in every language are among the most important preliminaries for conducting various multimedia researches such as talking head, lip reading, lip synchronization, and computer assisted pronunciation training applications. With respect to the fact that analyzing visemes is a language dependent process, we concentrated our research on Persian language, which indeed has suffered from the lack of such study. To this end, we proposed an image-based approach which consists of four main steps, including (i) extracting the lip region, (ii) extracting Interlaced Derivative Patterns (IDP) considering coarticulation effect, (iii) using a hierarchical approach for clustering visemes in the Persian language by mapping each viseme into its subspace, and finally (iv) applying a Support Vector Machine (SVM) to classify visemes which their classes have been obtained in the previous step. In order to clustering visemes, we applied unweighted pair group method with arithmetic mean to each feature vector. Then, furthest neighbor of the weight value as a result of reconstruction is set as a criterion for comparing viseme dissimilarity in order to find appropriate clusters. Afterwards, obtained clusters have been considered as the classes to which phonemes should be classified. In order to indicate the robustness of the proposed algorithm, a set of experiments was conducted on AVA in which two syllables were examined. Comparing the results of the clustering and classification algorithms, regarding the extracted features, with that of the perceptual test given by an expert proves a reasonable evaluation of the proposed algorithms.